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ABSTRACT 

Two program evaluations are described to demonstrate 
two very different methodologies and consumer orientations. The 
first, a patient satisfaction survey commissioned by a medical 
center, used surveys as part of a larger continuous multipurpose 
evaluation process, and the second, the ZooGardens eVw^luation, was a 
one-time multimethod evaluation of a small interagency program. In 
the patient satisfaction survey quarterly reports were reviewed and 
administrators were interviewed to help develop the surveys. The 
ZooGardens evaluation, which was requested by the Birmingham 
(Alabama) Park and Recreation Board, used both quantitative and 
qualitative methods, drawing on interviews, surveys, observations, 
document analysis, and daily attendance records. The characteristics 
that seemed to increase the positive impact of both evaluations and 
defined them as real and practical include the following: (1) 
timeliness; (2) responsiveness to client perceived needs; (3) 
responsiveness to client focus; and (4) flexibility. In both of these 
evaluations working relationships with program managers have been 
excellent, a factor cont r ibut ing to positive impact . Four tabl es 
summarize some evaluation findings. (Contains 6 references.) (SLD) 
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INTRODUCTION 

Evaluations corns in a variety of sizes / configurations and components 
from summative to fomative in nature and from number crunching to Gubanistic 
in perspective. Current debates encompass issues such as: do we judge 
programs and goals or do we just crunch the numbers; and how do we Integrate 
evaluation theory, the politics of change, and the selection of methodology to 
permit the best evaluation per program. The best usually meaning most 
understandable and useful to the consumer. However, as these debates rage, 
when our clients implement evaluation results and recommendations, when our 
evaluations change, focus, strengthen or even (however remote the 
possibility!) when they lead to the termination of a program, we have already 
been working within a very clear framework* One representation of this 
framework is what Morrell (1992) defines as values. Values that, he feels, 
tie the diverse field of evaluation into a somewhat cohesive whole. These 
values form the basic for describing the evaluation implementations in this 
paper. They are: 

1. Research should focus on assessing actions that can have 
practical consequences in real worlds settings. 

2. Research in all phases of its life cycle - from design to 
reporting of results, should aid decision and policy makers. 

3. The political nature of programs have legitimate but not 
exclusive claims on the conduct of evaluation. 

In other words, evaluations need to be practical and real, helpful, and 

politically responsive and in-tune. The purposes of this session are; 

1. describe how clients have implemented the results and the 
recommendations of two distinctly different evaluations and 

2. analyze the evalu^Ltions and identify the characteristics that 
seem to facilitate the utilization of evaluation results and 
recommendations . 

The big question is: what are the characteristics of these three values 
that lead to understandable and useful evaluations. 

The evaluations in this paper represent two very different methodologies 
and consumer orientations. The first utilizes surveys as part of a larger 
continuous, multi-purpose evaluation process, and the second is a one time 
multi-method evaluation of a small interagency program. The methodology for 
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determining implementation for the Patient Satiefaction Survey was to review 
the Quarterly Reports, to interview the Director ot the Poll and the Research 
Coordinator of the Medical Center ae to the development and use of the survey* 
The Research Coordinator was also asked specifically how the surveys impact on 
resources, program planning and program operations. For the Zoogardens 
evaluation the Recreation Superintendent was interviewed. In addition to a 
specific review of implementations based on each result and recommendation^ 
interview questions were; 

1. How were evaluation results disseminated and to whom? 

2. Where in the organization were program decisions made and how were these 
implemented? 

3. What were the criteria for implementing or rejecting a recommendation? 

4. What role does the evaluation play in future decision making and program 
direction? 

5. Did the evaluation lead to other changes not addressed within the 
evaluation? 



PATIENT SATISFACTION SURVEY 
The first ©valuation is a Patient Satisfaction Survey commissioned 
by a large regional Medical Center located in a major university town. The 
Medical Center is one of the largest employees in the area and has the busiest 
emergency room in the state. The traditional methodology for this survey, now 
in its 5th year, has been telephone interviews from a random sample of 
patients discharged each month* All interviews are conducted by experiencv^d, 
trained telephone interviewers using a computer assisted telephone (CATI) 
system. There are four survey sets, one for each area of the Medical Center: 
Inpatient Services, Outpatient Services, Rehabilation, and Speech and Hearing. 
Each survey takes about 30 minutes based on what services the patient used. 
The total number of questions in the Inpatient survey is 285 and in the 
outpatient 246, though no patient would have all of the questions. Responges 



to questions are yes or no; or are ranked responses ranging from strongly 
disagree (1) to strongly agree (4) or not applicable <5); or from very poor 
(1) to excellent (4), depending on the question. An overall grand mean, 
means, frequencies, and percentages aggregated by quarter are reported for 
each of the four areas in a Quarterly Report ♦ Also reported this year are the 
responses to the open ended questions that are included for some q[uestiona. 
Histograms and trend lines by month are also reported with each quarterly 
report. TABLES 1, and 2 below give a brief example of how collected data are 
reported in the Quarterly Reports. A separate Quarterly Report is provided 
for all four areas. 

Changes in the survey from year to year have generally originated from 
three sources: 

1. the results of previous questions have generated new questions 
or sources of interest or concern 

2 • new trends or environmental concerns have demanded new 
questions or the elimination of old ones and 

3. new departments and areas of the Medical Center have teen added 
to the survey. 

During the planning for the 1993 survey, for example, the evaluator was 
able to increase the use of the survey for the client by: 

1* expanding the categories of data collected to include all four areas of the 
Medical Center for the first time, 

2. providing more differentiated data by departments and 
3 expanding the open ended questions so that specific concerns could be 
addressed within each department related to patient satisfaction as part of a 
new, center wide. Quality Assurance Program* 

Quoting Fein, Staff and Kobylenski (1993, p. 9 ) "Productive findings 
that are used by program planners and managers has always been a major goal 
for evaluatore. *' They also argue that evaluation is '*an exercise in power 
within a program or organization" (1993 p«9). Quoting, "Because use of 
results often leads to program change of small or major magnitude, and program 
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DCH OUTPATIENT SERVICES 
AUGUST DISCHARGES, 1993 
SCHEDULING 



TABLE 1 





DISAGREE 


AGREE 


STRONGLY AGREE 


WALKMN 


COUNT 


PERCENT 


COUNT 


PERCENT 


COUNT 


PERCENT 


COUNT 


PERCENT 


ABLE TO SCHEDULE APPOINTMENT 


3 


1.4X 


172 


81.5% 


31 


14.7% 


5 


Z.4% 



Q-12 You were able to schedule an appointment for services within a 
reasonable period of time. 

(IF STRONGLY DISA&REE OR DISAGREE) Why do you feel this way? 



THEY TOOK TOO LONG TO GET STARTED ON MY THERAPY. 

THEY TOOK FOR GRANTED THAT I KNEW TO GO TO THE NEW BUILDING WHEN I DID 

NOT; THEY SPENT TOO MUCH TIME TRANSFERRING ME AROUND. 
THE DATES WERE MIXED SOMEHOW, 



TABLE 2 



DCH OUTPATIENT SERVICES 
AUGUST DISCHARGES, 1993 

WHAT SERVICE IMPRESSED YOU MOST FAVORABLY? 



ADMISSIONS/REGISTRATION 

THE ADMISSION PROCESS 
THE ADMISSIONS 

THE LOCATION OF ENTRANCE AND EXIT. THE ADMISSIONS STAFF WERE REAL 
NICE. 



CANCER CENTER 

THE CANCER CENTER. THEY TREAT YOU LIKE YOU ARE SPECIAL. 
THE cm % CENTER 

THE CANCcR TREATMENT CENTER. EVERYONE IS VERY CARING AND HELPFUL; NO 
PROBLEMS. 

THE QUALITY OF X-RAYS AND TECHNICIANS AT THE CANCER CENTER 



CARDIOLOGY 

I REALLY LIKED THE TECHNICIAN WHO DID MY EKG; SHE WAS REALLY .AST. 



FACILITIES/EQUIPMENT 

THE NEW OUTPATIENT FACILITY MAKES IT POSSIBLE TO HAVE EVERYTHING DONE 
IN ONE AREA INSTEAD OF HAVING TO RUN FROM FLOOR TO FLOOR TO RECEIVE 
SERVICES. 

THE SURROUNDINGS IN ADMISSIONS 

THE WHOLE ENVIRONMENT. THE ATMOSPHERE WAS COMFORTABLE ; RAN VERY 

EFFICIENTLY. 
THE SURROUNDINGS AND THE PERSONNEL WERE GREAT! 
IT WAS NICE AND NEW , I WAS PLEASED WITH EVERYONE. 
THE WAITING ROOM AND THE NICENESS OF THE PERSONNEL. 

IT IS A VERY NICE FACILITY . THE STAFF WAS FRIENDLY, AND WE WERE ABLE TO 

GET IN AND OUT QUICKLY. 
THE NEW FACILITIES WERE GREAT. 
THE FACILITY IS NEW AND CLEAN. 
PLEASANT SURROUNDINGS AND PROMPTNESS 

THE LOCATION OF ENTRANCE AND EXIT . THE ADMISSIONS STAFF WERE REAL 
NICE. 

THE EXCELLENT FACILITIES 

I WAS IMPRESSED WITH THE FACILITIES; IT WAS QUIET AND RELAXING. 

THE ATMOSPHERE WAS PLEASANT. 

THE BUILDING IS NICE; IT MAKES YOU FEEL GOOD. 

THE NEW PART : THEY WERE VERY PROMPT. 
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change in an organization is an exercise in power r tha fullest understanding 
of utilization of evaluation findings must consider how change is effected 
and how evaluation is related to power** (1993 p»10). This "power** or what I 
prefer to call "client empowermerit " results from evaluations that are: 
Practical and real; Helpful; and Politically responsive and in-tune. 

Evaluations can effect; 1. policy, 2. program planning and 
implementation and 3. program operations. This is clearly seen in the Medical 
Center's use of responses to the open ended questions the **literals". 

These **literal8" are reported back to the Medical Center *s monthly and 
are used as the basis for staff development and program improvement within 
each unit* The procedure at the Medical Center is to disseminate the 
pertinent section of the report to each of tne members of a unit and to 
discuss the results in a staff meeting* As an example of how the Medical 
Center utilizes the results, housekeeping has complained for several years 
that weekend staffing is a problem but they could not get approval for 
additional personnel until they were able to take the data provided which 
clearly indicated that patients noticed housekeeping was not performed on 
weekends. With the data for three months, housekeeping submitted a proposal 
to the administration and received a per diom for weekend housekeeping. 

Additionally, the Medical Center is able to take the data they collect 
on patient charts which clocks the actual time patient q wait for services, or 
spend in the emergency room and then, with the poll data, they can compare the 
actual versus the patient perceived waiting time. The emergency room has 
established a Patient Satisfaction Task Force to address this and other 
concerns related to patient satisfaction. 

When the Out Patient Center was opened there were complaints about the 
parking so within the next survey questions were added that specifically 
addressed the parking issue and lead to how the parking for this center was 
developed and implemented. 

The Research Coordinator specifically mentioned that because the surveys 
are center wide, each department gets an unbiased, consistent report. Since 

6 
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May of 1993 until this past weok she had coropxeted almost 70 presentations to 
departments and units for ataff development and Quality Improvement, with 
Health Reform already in the process/ the center is already able to respond to 
inquiries from insurance companies about charges and the length of patient 
stays, and if there are other issues that this center is addressing that is in 
the development stages elsewhere. The resuxts from the Quarterly Reports also 
go into the center's Annual Reports and other internal research projects. 
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Zoogardens Summary 

The ZooGardens Evaluation waa conducted at the request of the Recreation 
Division of the Birmingham Park and Recreation Board and was a formative 
evaluation of the ZooGardene Project, which has been a unique, cooperative 
undertaking of three entities: the Hecreation Division of the Birmingham Park 
and Hecreation Board, the Birmingham Zoological Society, and the Birmingham 
Botanical Gardens* 

ZooGardona originated in 198& with the Recreation Division and the 
Zoological Society and was designed to provide an environmental awareness 
project for city children between the ages of 8 and 10. In 1989 the Botanical 
Gardens joined the venture. The summer of 1991 was the fourth year for the 
project which continued to target children 8-10 years old who participated in 
three days of special activities, games, and tours at the Zoo and at the 
Botanical Gardens. 

Children have been enrolled for this project primarily though 
participation in summer camps held at the 17 city recreation centers under the 
Hecreation Division (see TABLE 3 for a summary of project staff). The major 
expenses have been: transportation to and from Lane Zoo, which has been 
supplied by the Recreation Division; materials and supplies for crafts, which 
have been provided by each division; and a Zoological Society staff person, 
who has been paid by the Zoological Society. 



TABLE 3* 
PROJECT STAPF 



ORGANIZATIONAL CHART 


RECREATION DIVISION 

* Recfctttioa Supervifocs (1) 

■ IUcimUoo CeelCT Duvxtonfl 7) 

* IUcimUcb CMcr Imdm (17) 


BOTANICAL 0AM5ENS 
* Diiecior of Etuoukn (1) 
•1- Simiot Lmtfrn VoVrntmn (6) 


ZOOLOGICAL sociery 

-VohxoMcf CoofdiiMlor (1) 

* AMMUnC Siamer Volupl«er CoordlmtcH <1) 

* Zooonqp CoofdiiMtor (1) 
•1- Tmo CoudmIot* C2I) 

i- TMdw Halficr (1) 



* paid staff 
+ volunteer staff 
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1. The project hae an average total cost of $20*36 per child. 
ATTENDANCE 

1 . There are probleme with attendance and these problems 
result from the enrollment process « 

A. Current Attendance is at about 73% of recommended attendance and 
better enrollment procedures would increase attendance. 

B. The average attendance across weeks is inconsistent. 

2 • Dropout during the program is not a primary problem, though there are 

some children who attend for only one or two days. 
3. Younger and cider children are still a concern, but not as 

critically as in the past. 



Recommendations 

There are three categories of recommendations that resulted from this 
evaluation, the first pertain to the need to sophisticate and organize the 
enrollment and recruitment process; the second category concerns program 
suggestions and the third category of recommendations outline possible future 
evaluations focusing on outcome measure of children' attitudes utilizing a 
more quantitative approach. 
CATEGORY ONE; Enrollment and Recruitment 

1. Pre-reqister for ZooGardens 

2. Involve Zoo and Potanical Gardens more in enrollment process 

Although it is physically easier for volunteers to keep track of only 3- 
4 children in a group, and the children get a great deal of individualized 
attention in such small groups, there would be little lost in quality, in 
individualized attention, or in group size with more children attending the 
sessions. Center directors often mentioned that one of the things they liked 
best about the project was the small groups which allowed for the 
individualized attention. However, the maximum capacity of between 30 to 40 
children a day was based on the capacity of the bus used for transportation 
and with this maximum in mind, each location, the Zoo and the Botanical 
Gardens, designed their programs and deliver/ systems to meet this capacity. 
There have been sufficient teen counselors and Botanical Garden volunteers to 
maintain the small groups that the centers find valuable for the children, but 
these volunteers have not been utilized to the fullest potential because of 

10 
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the less than cnaxinmm enrollments. Although increasing enrollment is a 
concern/ of even greater concern is the inconaiatent enrollment. 

Personnel are available, such as the Zoocamp Coordinator and the 
Botanical Garden voluntsers, who could help to coordinate the recruitment 
process by meeting with the centers in May and/or June to help prc-register 
children for this program. These people might take on a role such as helping 
with the promotion of the project, as a contact with the centers to make sure 
children are pre-registered, or with helping to train the directors and 
leaders better about the project so they can be more involved in the 
activities with the children both before and after ZooGardens. It would be 
difficult for these ZooGardens staff members to be primarily involved with the 
registration of ZooGardens as the Junior League Volunteers are usually not 
available until May at the earliest and the Zoocamp Coordinator does not begin 
working with ZooGardens until school is out, during the first week in June. 
However, they have been identified by project managers as one resource that is 
available and that could be utilized in the enrollment process. 

3» Engage use of center directors who have a strong commitment to this project 
to help coordinate enrollment procedures on the centers end 

As with any other program offered through the centers, directors have 
the responsibility to function "as a public relations agent for the 
Birmingham Park and Recreation Board in his center's community. This involves 
contacting civic clubs, P.T.A.s, and other interested groups about the Center 
programs....".^ The directors, however, have incorporated ZooGardens into 
their summer camp program as a regularly scheduled field trip. This has borh 
benefits and drawbacks* The benefits are that children are available and arc 
organized for this activity. The drawbacks are several: kids are left behind 
because they are the incorrect age to participate; children in families are 
separated; directors need to substitute an equivalent activity for the other 



^Birmingham Park and Recreation Board. (1988). Recreation Center Handbook , 
page 17. 
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children and often cannot do this because of the lack of transportation; and 
sometimes directors need to restructure personnel use to provide supervision 
for ZooGardens children and the remaining children at the center. 

It is not feasible to centralize or even tightly coordinate the 
ZooGardens enrollment process because of the distance between centers and 
because of the lack of technical and electronic management systems which would 
be necessary to coordinate or centralize enrollment. However, there are 
several directors, who because of their strong commitment to this program, 
might be interested in exploring how the centers are scheduled to attend, 
alternatives to grouping the centers for a more consistent enrollment, and who 
might have ideas on increasing enrollment and who would have valuable input 
into improving the process. 

4« Rearrange time _for ZooGardena to better follow pattern of children 

attending the centers. 

Many children attend the centers when they do because of the concurrence 
of a summer lunch program and the summer camps offered at the centers. This 
concern was addressed during the presentation of the ZooGardens Evaluation 
Final Report. ZooGardens is tentatively scheduled to begin one week earlier 
next summer, and will not have a break during the week of July 4th as the 
holiday will not affect the day of camp. This will make the schedule coincide 
with the pattern of attendance at the centers for maximum possible 
enrollments . 

CATEGORY TWO; Program development and Implementation 

Other issues that were identified were the distraction of younger or 
older children who had been inappropriately included in the group, or who 
tended to follow the zoo campers while not actually joining in the activities. 
This did not occur as consistently as in past years, but was a probler on some 
days. During the 1991 session there were children as young as two years old 
and as old as 12 years old who were brought by the centers. More efficient 
enrollment should help this situation, other suggestions made through 

12 
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interviews throughout the evaluation aro: 

X* Offer a separate one day program for younger children, which would adapt 
ZooGardens specifically for ages 4-7 ( or other identified ages). 

2* Give the children awards or certificates on the last day for their 

participation in ZooGardens. This would be something tangible that the 
child could take back indicating the nature of the program* 

3« In order to better prepare the children for the ZooGardens experience and 
to provide for follow*-up, it was suggested that there be a pre/and or post 
visit to the center to reinforce the concepts* This would provide for an 
extended use of the children's notebooks and a way to extend the learning 
experience* This would be a way to also involve the directors more in the 
ZooGardens process* Activities, such as grow labs, are available 
which could be completed at the centers and which would reinforce the 
projects goals and objectives. 

4* The education coordinators have continued to develop activities and games, 
as a part of the continuing development of ZooGardens as evidenced by the 
Planned addition on Alabama Indians at the Botanical Gardens for next year 
and the addition of the Outdoor Learning Centers at the Zoo this year. 
This is important because children do repeat the program and even though 
they repeat because they liked what was already available, the additions 
will help to keep their continued interest and provide new information. 

Category Three; Suggestions for future Evaluations 

The one major area that needs to be evaluated more closely is the 

outcome of the program on the children attending this program* Initial 

indications are that the children do learn from this project and their 

involvement with it, but a key indicator would be a quantitative measure. 

There are tests that have been developed that are specifically environmentally 

oriented. However they are generally far to broad in scope to apply to this 

project or they are to narrow in topic* What would be appropriate would be a 

measure based on the content of this project specifically* This would be 

difficult as some flexibility in content coverage is a part of the natural 

design of this project* However several techniques could be applied to this 

project in the development of a measure that could provide a better indication 

of program effect* 

The first would be to develop a list of the critical vocabulary that are 
emphasized in this project, such as transpiration, ecology, habitat* Pictures 
could be used to define each word and a 10 - 20 item vocabulary matching test 
could be developed* 

In addition to vocabulary, the concept of prior knowledge could be 
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applied in the development of a test instrument. For example, children could 
be asked to draw specific situations, scenes, or animals in their appropriate 

habitat. A situation might be •'can you show me how occurs**, or •'draw 

you favorite animal", or "draw an animal that lives in the desert", etc. One 
or two drawings per child could be evaluated by a measure of the amount of 
detail or accuracy. The same questions would be used for a post test to 
obtain a measure of change. 

IMPLEMENTATION 

The process for improving the implementation of Zoogarden's results and 
recommendations included: 

1. Consistent confirmatory interviews throughout the data collection process 

2. Oral presentation of preliminary findings with opportunity for discussion 
and input from the project supervisors 

3. A focus on the critical change points identified through the data 
collection and analysis and supported by both one and two above. 

One particular point addressed in the evaluation, was the need to 
involve the center directors better. The Supervisor said that once the center 
directors were involved, they understood the project better. Supervisors went 
over the objectives with the directors during the planning stages, and though 
they had in the past attempted to tell the directors the objectives, the 
directors had not understand the program and were very critical of it. With 
increased understanding and with a sense of ownership, the directors' 
criticism had diminished significantly. The center directors were involved 
with the planning from the beginning and had a much stronger interest in the 
program. The Zoo and the Botanical Garden staffs also attended the staff 
meetings for the center directors and were able to get a better perspective of 
their concerns and problems. This lead to a stronger relationship [p between 
the two and a much stronger, coordinated program for the children. They will 
continue to do this each year. 

Other examples are included in the TABLE 4 which lists each 
recommendation and comments for each of the two following years. 

14 
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TABLE 4 

ZOOGARDENSSUMKtARY 1992-93 



RBCmiMENDATIONS 


1992 


1993 


CATEGORY ONE: ENROLLMENT AND RECRUTTMENT 


1 . PRE-REOI^rER FOR ZOOGARDENS 


Tne4 to do thii, with tosat wcceu. 


Worked out well this year. Lead to 
more enthusiasm on the part of the kids. 
They will continue to do this. 


2. INVOLVE ZOO AND BOTANICAL 
GARDENS MORE IN ENROLLMENT 
PROCESS 


TtUced about and planned, but did not 
implement. 


Worked very well, the staff went to the 
cenien to prepare the kids for the Zoo 
and the Botanical Gardens. Everyone 
loved it. 


3. ENGAGE USE OF CENTER 
DIRECTORIES WHO HAVE A STRONG 
COMMrrMENT TO THIS PROJECT TO 
HELP COORDINATE ENROLL>.4ENT 
PROCEDURES ON THE CENTERS END 


Could no< coordixute it thia year; 
centralized enrolment ia not feasible 
becauae of the layout of centers and it ia 
not appropriate here. 


Directon were involved from the 
beginning in the planmng and it nude 
all the difference in the world. They 
will continue thia with new directors 
added each year, while still retaining 
some of the old ones on the planmng 
committee. 


4. REARRANGE TIME FOR 
ZOOGARDENS TO BETTER FOLLOW 
PATTERN OF CHILDREN ATTENDING 
THE CENTERS 


Implemented at end of program during; 
the summer of 1991 . One week wai 
eliminated and centers were combined 
for a better enrollment number. 


Followed the same schedule as 1992, 
with great improvement tn enrollment 
marugemcnt. 


CATEGORY TWO: PROGRAM DEVELOPMENT AND IMPLEMENTATION 


1 . ELIMINATE OR ACCOMMODATE 
YOUNGER CHILDREN 
A. SEPARATE PROGRAM 
E. AWARDS 

C. PRE/POST VISITS TO CENTERS 

D. CONTINUED CURRICULUM 
DEVELOPMENT 


They tried to do thia by elimiiuting 
younger children, but the center directors 
were very vocal about the need to 
acc ommodate younger children in the 
1993 meeting. 


The programa are being adapted for 
younger children^ they are less rigid and 
it haa worked very well this year. Both 
the Zoo and the Botanical Gardens have 
developed new programs or parts of 
program each year. 

The pre visits to the center have been s 
key to the continued development of this 
program. One important point was the 
addition of a very supportive person to 
do the program for the Zoo. 

Though ihey have not given awards or 
certificatea, they have not ruled this out. 
Other things are given to the children at 
thia time, such aa things they malce. 


CATEGORY THREE: SUGGESTIONS FOR FUTURE EVALUATIONS 


I . PROGRAM EVALUATION 


They feel that an evaluation every 4-5 yean would be great. It ukea time to change 
people and to get people on board to do this kind of a program and by the third year 
they are almost ready to see what else they need to work on. 

They would ipecifically like to took at the environmental aspects of the program and 
what the children are learning. 
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ISTCOFVAVAiLABLE 



The characteristics that seem to increase the positive impact of 
evaluation results and define our evaluations as practical and real, helpful 
and politically responsive and in-tune are: tiroelinesB (a great standard for 
asBeasing evaluations), responsiveness to client perceived needs, responsive 
to client focus, and flexibility. Evaluate ns are useful if they meet the 
client where they are and help them identify what they want and need. 

Sanders (1993), though addressing the use of evaluations in foundations^ 
hit the nail on the head for hundreds of other businesses, institutions, 
agencies, boards, etc. who also want programs to meet specific needs and reach 
specified goals, when he says "evaluations must be useful and feasible in 
addition to being as accurate ae they can be." (1993 p.38) • One of his 
suggestions is to work with program managers throughout the entire process, 
the study design, the progress, the program findings. Throughout both of 
these evaluations, with working relationships of differing degrees of 
formality, this has clearly bee', a strong point. Each evaluation is 
different, but unquestionably these characteristics seem to increase the 
positive impact of evaluation results and recommendations by helping clients 
make decisions about policy, resources, program planning and operations. One 
interesting note, change takes time, and it is accomplished one step at a time 
and sometimes one program's steps may be bigger than another's, but it ia the 
process of change and improvement that is critical • 
What more can we ask for? 

********************************************** 



ACKNOWLEDGEMENTS : 

Special thanks to Dr. Dobra McCallum at The Capstone Poll, Institute for 
Social Science Research, University of Alabama, for providing illustrative 
examples of Poll results for use throughout this paper. 



16 

19 



BIBLIOGRAPHY 



Fein, E. , Staff, I., Cl Kobylenski S. (1993). The evaluator as power merchant. 
Evaluation Practice . 14(1). 9-16. Greenwich, Connecticut: JAI Press Inc. 

Leonard, E. C. Jr., & Miller, M. J. <1992). The role of evaluation in 
increasing the usefulness of employee involvement programs. Evaluation 
Practice . 13(1). Greenwich, Connecticut: JAI Press Inc. 

Morell, J. A. (1992). Industrial productivity: New perspectives for a 
traditionally trained evalwator. Evaluation Practice . 13(2) .125-134. 
Greenwich, Connecticut: JAI Press Inc. 

Sanders, J. R. (1992). Evaluations in foundations, commentB, Evaluation 
Practice . 13(1). Greenwich, Connecticut: JAI Press Inc. 

Weiss, C. H. (1993). Where politics and evaluation research meet. Evaluation — 
Practice . 14(1). 93-106. Greenwich, Connecticut; JAI Press Inc. 

Weiss, C. H. (1993). Politics and evaluation: a reprise with mellower 
overtones. Evaluation Practice . 14(1). 107-109. Greenwich, Connecticut: 

JAI Press Inc. 



17 



